Post-selection Inference in Multiverse Analysis (PIMA): An Inferential Framework Based on the Sign Flipping Score Test.

When analyzing data, researchers make some choices that are either arbitrary, based on subjective beliefs about the data-generating process, or for which equally justifiable alternative choices could have been made. This wide range of data-analytic choices can be abused and has been one of the underlying causes of the replication crisis in several fields. Recently, the introduction of multiverse analysis provides researchers with a method to evaluate the stability of the results across reasonable choices that could be made when analyzing data. Multiverse analysis is confined to a descriptive role, lacking a proper and comprehensive inferential procedure. Recently, specification curve analysis adds an inferential procedure to multiverse analysis, but this approach is limited to simple cases related to the linear model, and only allows researchers to infer whether at least one specification rejects the null hypothesis, but not which specifications should be selected. In this paper, we present a Post-selection Inference approach to Multiverse Analysis (PIMA) which is a flexible and general inferential approach that considers for all possible models, i.e., the multiverse of reasonable analyses. The approach allows for a wide range of data specifications (i.e., preprocessing) and any generalized linear model; it allows testing the null hypothesis that a given predictor is not associated with the outcome, by combining information from all reasonable models of multiverse analysis, and provides strong control of the family-wise error rate allowing researchers to claim that the null hypothesis can be rejected for any specification that shows a significant effect. The inferential proposal is based on a conditional resampling procedure. We formally prove that the Type I error rate is controlled, and compute the statistical power of the test through a simulation study. Finally, we apply the PIMA procedure to the analysis of a real dataset on the self-reported hesitancy for the COronaVIrus Disease 2019 (COVID-19) vaccine before and after the 2020 lockdown in Italy. We conclude with practical recommendations to be considered when implementing the proposed procedure.

Interpersonal motor synchrony in autism: a systematic review and meta-analysis.

Introduction: Interpersonal motor synchrony (IMS) is the spontaneous, voluntary, or instructed coordination of movements between interacting partners. Throughout the life cycle, it shapes social exchanges and interplays with intra- and inter-individual characteristics that may diverge in Autism Spectrum Disorder (ASD). Here we perform a systematic review and meta-analysis to summarize the extant literature and quantify the evidence about reduced IMS in dyads including at least one participant with a diagnosis of ASD. Methods: Empirical evidence from sixteen experimental studies was systematically reviewed, encompassing spontaneous and instructed paradigms as well as a paucity of measures used to assess IMS. Of these, thirteen studies (n = 512 dyads) contributed measures of IMS with an in situ neurotypical partner (TD) for ASD and control groups, which could be used for meta-analyses. Results: Reduced synchronization in ASD-TD dyads emerged from both the systematic review and meta-analyses, although both small and large effect sizes (i.e., Hedge's g) in favor of the control group are consistent with the data (Hedge's g = .85, p < 0.001, 95% CI[.35, 1.35], 95% PI[-.89, 2.60]). Discussion: Uncertainty is discussed relative to the type of task, measures, and age range considered in each study. We further discuss that sharing similar experiences of the world might help to synchronize with one another. Future studies should not only assess whether reduced IMS is consistently observed in ASD-TD dyads and how this shapes social exchanges, but also explore whether and how ASD-ASD dyads synchronize during interpersonal exchanges.

Mood moderates the effects of prefrontal tDCS on executive functions: A meta-analysis testing the affective state-dependency hypothesis.

BACKGROUND: In recent decades, numerous studies have investigated the effects of transcranial direct current stimulation (tDCS) on cognitive functioning. However, results of these studies frequently display inconsistency and pose challenges regarding replicability. The present work aimed at testing the hypothesis of mood as potential moderator of prefrontal tDCS effects on executive functions (EF). This hypothesis refers to the relationship between mood and EF, as well as to the association of mood with the dorsolateral prefrontal cortex (dlPFC) activity. METHODS: We conducted a meta-analysis of 11 articles where the dlPFC was stimulated with anodal tDCS, EF were measured, and mood was assessed prior to the stimulation. We then conducted a meta-regression to examine whether mood moderated the tDCS effects on EF. RESULTS: While no significant effect of tDCS on EF emerged from the meta-analysis, the meta-regression indicated that mood plays a significant role as moderator, with greater tDCS effects on EF in individuals with higher depressive symptoms. LIMITATIONS: The limited number of studies included, the heterogeneous samples considered, and the limited generalizability to other non-invasive brain stimulation techniques and affective states. CONCLUSIONS: Findings suggest that evaluating mood prior to stimulation could increase the sensitivity and specificity of tDCS application, and provide the first meta-analytic evidence in favor of the affective state-dependency hypothesis.

First steps into the pupillometry multiverse of developmental science.

Pupillometry has been widely implemented to investigate cognitive functioning since infancy. Like most psychophysiological and behavioral measures, it implies hierarchical levels of arbitrariness in preprocessing before statistical data analysis. By means of an illustrative example, we checked the robustness of the results of a familiarization procedure that compared the impact of audiovisual and visual stimuli in 12-month-olds. We adopted a multiverse approach to pupillometry data analysis to explore the role of (1) the preprocessing phase, that is, handling of extreme values, selection of the areas of interest, management of blinks, baseline correction, participant inclusion/exclusion and (2) the modeling structure, that is, the incorporation of smoothers, fixed and random effects structure, in guiding the parameter estimation. The multiverse of analyses shows how the preprocessing steps influenced the regression results, and when visual stimuli plausibly predicted an increase of resource allocation compared with audiovisual stimuli. Importantly, smoothing time in statistical models increased the plausibility of the results compared to those nested models that do not weigh the impact of time. Finally, we share theoretical and methodological tools to move the first steps into (rather than being afraid of) the inherent uncertainty of infant pupillometry.

Entia Non Sunt Multiplicanda Shall I look for clusters in my cognitive data?

Unsupervised clustering methods are increasingly being applied in psychology. Researchers may use such methods on multivariate data to reveal previously undetected sub-populations of individuals within a larger population. Realistic research scenarios in the cognitive science may not be ideally suited for a successful use of these methods, however, as they are characterized by modest effect sizes, limited sample sizes, and non-orthogonal indicators. This combination of characteristics even presents a high risk of detecting non-existing clusters. A systematic review showed that, among 191 studies published in 2016-2020 that used different clustering methods to classify human participants, the median sample size was only 322, and a median of 3 latent classes/clusters were detected. None of them concluded in favor of a one-cluster solution, potentially giving rise to an extreme publication bias. Dimensionality reduction techniques are almost never used before clustering. In a subsequent simulation study, we examined the performance of popular clustering techniques, including Gaussian mixture model, a partitioning, and a hierarchical agglomerative algorithm. We focused on their ability to detect the correct number of clusters, and on their classification accuracy. Under a reasoned set of scenarios that we considered plausible for the cognitive research, none of the methods adequately discriminates between one vs two true clusters. In addition, non-orthogonal indicators lead to a high risk of incorrectly detecting multiple clusters where none existed, even in the presence of only modest correlation (a frequent case in psychology). In conclusion, it is hard for researchers to be in a condition to achieve a valid unsupervised clustering for inferential purposes with a view to classifying individuals.

The role of raters threshold in estimating interrater agreement.

The evaluation of agreement among experts in a classification task is crucial in many situations (e.g., medical and psychological diagnosis, legal reports). Traditional indexes used to estimate interrater agreement (such as Cohen's j) simply count the number of observed agreements and correct them by removing chance agreements. In this article, we introduce a new theoretical framework for the evaluation of interrater agreement based on the possibility of adjusting the observed classifications conducted by the raters. This framework refers to the introduction and formalization of two concepts involved in the classification task: (a) the belonging measure of an object to a category and (b) the rater's belonging threshold, which is the minimally sufficient value of the belonging measure at which the rater will classify an object into a category. These factors are ignored by traditional indexes for interrater agreement, though their role may be decisive. Two Bayesian models are tested through a Monte Carlo simulation study to evaluate the accuracy of the new methodology for estimating raters' threshold and the actual degree of agreement between two independent raters. Results show that the computation of traditional indexes for interrater agreement on the adjusted classifications leads to a more accurate estimation of the experts' actual agreement. This improvement is greater when a large difference between raters' belonging thresholds is observed; when the difference is small, the proposed method provides similar results to those obtained in the simple observed classifications. Finally, an empirical application to the field of psychological assessment is presented to show how the method could be used in practice. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Heterogeneity and Factorial Structure in Alzheimer's Disease: A Cognitive Perspective.

BACKGROUND: Alzheimer's disease (AD) patients show heterogeneous cognitive profiles which suggest the existence of cognitive subgroups. A deeper comprehension of this heterogeneity could contribute to move toward a precision medicine perspective. OBJECTIVE: In this study, we aimed 1) to investigate AD cognitive heterogeneity as a product of the combination of within- (factors) and between-patients (sub-phenotypes) components, and 2) to promote its assessment in clinical practice by defining a small set of critical tests for this purpose. METHODS: We performed factor mixture analysis (FMA) on neurocognitive assessment results of N = 230 patients with a clinical diagnosis of AD. This technique allowed to investigate the structure of cognitive heterogeneity in this sample and to characterize the core features of cognitive sub-phenotypes. Subsequently, we performed a tests selection based on logistic regression to highlight the best tests to detect AD patients in our sample. Finally, the accuracy of the same tests in the discrimination of sub-phenotypes was evaluated. RESULTS: FMA revealed a structure characterized by five latent factors and four groups, which were identifiable by means of a few cognitive tests and were mainly characterized by memory deficits with visuospatial difficulties ("Visuospatial AD"), typical AD cognitive pattern ("Typical AD"), less impaired memory ("Mild AD"), and language/praxis deficits with relatively spared memory ("Nonamnestic AD"). CONCLUSION: The structure of cognitive heterogeneity in our sample of AD patients, as studied by FMA, could be summarized by four sub-phenotypes with distinct cognitive characteristics easily identifiable in clinical practice. Clinical implications under the precision medicine framework are discussed.

Stereotype threat effects on Italian girls' mathematics performance: A failure to replicate.

Many studies have found that males, on average, perform better than females in mathematics, although the size of this gender gap is small and varies considerably across countries. Stereotype threat has been proposed as a principal cause of this gender gap. From this perspective, females' performance is affected by fear of confirming a negative stereotype about females' mathematical ability and this stereotype can be activated by an experimental manipulation that reminds females of the stereotype. Yet, evidence of a stereotype threat effect on mathematics performance in childhood and adolescence has been mixed. The present study replicated a highly cited study of stereotype threat among Italian adolescents with a much larger sample of Italian ninth grade (89 male, 75 female, mean age = 14.2) and eleventh grade (84 male, 80 female, mean age = 16.2) public high school students. Performance in tests administered both before and after the experimental manipulations were analyzed with a series of logistic mixed-effects models. Model comparisons confirmed that males performed better than females, but the probability of a stereotype threat effect was infinitesimal. We conclude that Italian adolescent gender differences in mathematics may not be explained by stereotype threat effects. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Development and Psychometric Properties of a New Self-Report Questionnaire Measuring Attachment in School-Age Children: The Attachment in Middle Childhood Questionnaire (AMCQ).

Although conscious aspects of attachment representations can be effectively assessed in middle childhood, the few available self-reports are based on different operationalizations of attachment and do not always show adequate psychometric properties. The current study aimed to develop and evaluate the psychometric properties of the Attachment in Middle Childhood Questionnaire (AMCQ). Using three independent samples of Italian children (Mage = 10 years, 52% = girls), in three interrelated studies we (1) selected a pool of items from existing questionnaires, adapted them to the same response format, and subjected them to exploratory factor analysis; (2) performed confirmatory factor analyses on the retained items; and (3) used structural equation modeling to assess the factor structure, external validity, and invariance across gender and age groups. The final 15-item questionnaire comprised two dimensions (anxiety and avoidance) and a supplementary scale (security). Overall, results supported the reliability and validity of the AMCQ for Italian children.

Data and supplemental material of the paper "Effectiveness of digital-based interventions for children with mathematical learning difficulties: A meta-analysis".

Data and supplement material of the article "Effectiveness of digital-based interventions for children with mathematical learning difficulties: A meta-analysis" (Benavides-Varela et al.) [1] are presented. Data were collected from studies included in the meta-analysis to evaluate the effects of digital-based interventions for children with mathematical learning difficulties compared to control conditions in group-designed randomized controlled trials. Literature search, inclusion criteria and coding procedure are described. PRISMA flow-chart is reported to summarize the literature search and coding of all the relevant characteristics of the primary studies is made available. This allows other researchers to easily access to the information needed to evaluate the studies and to use these data in future meta-analyses. However, researchers are highly recommended to refer to the original papers in order to check studies suitability to their own criteria. Moreover, in the supplemental material all the information needed to reproduce the meta-analysis results is reported together with the R code syntax. Data and supplemental material are available online (https://osf.io/ajdnv/).

False memory for pictorial scripted material: the role of distinctiveness and negative emotion.

Emotional content has complex effects on false memory. Under certain circumstances, emotional material may reduce the likelihood of false memory, a phenomenon that some consider related to it being more distinctive than neutral stimuli. In the present study we tested inferential false memory related to emotionally neutral or negative, and distinctive (but not emotionally charged) scripted material. Remember/familiar judgements were required for recognised stimuli. Data were analysed using mixed-effects multinomial regressions and a Bayesian inferential approach. Results obtained with 82 adult participants showed that, compared with neutral material: distinctive material reduced their false memory associated with "remember" and "familiar" judgements, virtually to the same extent; negatively-charged material reduced false memory associated with "remember" judgements but it had no effect on false memory associated with "familiar" judgements. In short, negatively-charged and distinctive material seems to affect false memory in different ways: the latter affects both recollection and familiarity, the former only recollection.

Proprioceptive accuracy in Immersive Virtual Reality: A developmental perspective.

Proprioceptive development relies on a variety of sensory inputs, among which vision is hugely dominant. Focusing on the developmental trajectory underpinning the integration of vision and proprioception, the present research explores how this integration is involved in interactions with Immersive Virtual Reality (IVR) by examining how proprioceptive accuracy is affected by Age, Perception, and Environment. Individuals from 4 to 43 years old completed a self-turning task which asked them to manually return to a previous location with different sensory modalities available in both IVR and reality. Results were interpreted from an exploratory perspective using Bayesian model comparison analysis, which allows the phenomena to be described using probabilistic statements rather than simplified reject/not-reject decisions. The most plausible model showed that 4-8-year-old children can generally be expected to make more proprioceptive errors than older children and adults. Across age groups, proprioceptive accuracy is higher when vision is available, and is disrupted in the visual environment provided by the IVR headset. We can conclude that proprioceptive accuracy mostly develops during the first eight years of life and that it relies largely on vision. Moreover, our findings indicate that this proprioceptive accuracy can be disrupted by the use of an IVR headset.

Can Mentoring Promote Self-esteem and School Connectedness? An Evaluation of the Mentor-UP Project / ¿Puede la mentoría fomentar la autoestima y la conexión escolar? Evaluación del proyecto Mentor-UPP

Research in the United States has shown that youth mentoring is a promising strategy for increasing self-esteem and school connectedness in at-risk youth. There has been little confirmation of those findings internationally. The current study evaluates the impact of mentoring by trained university students on children's self-esteem and school connectedness compared to schoolmates not involved in the program. Mentor-UP is a school- and community-based weekly mentoring program implemented in northern Italy over a period of seven months. Participants (209 students - 34 in the experimental group and 175 in the comparison group - aged between 11 and 13, 56% male, 27% immigrants) reported their levels of self-esteem and school connectedness at the beginning and at the end of the program. Results showed a significant increase in mentees' self-esteem compared to the control group, while the difference in school-connectedness was nonsignificant. The findings support the effectiveness of Mentor-UP in nurturing youth's self-esteem

La investigación en EE. UU. ha demostrado que la mentoría juvenil es una estrategia prometedora para aumentar la autoestima y la conexión escolar en jóvenes en situación de riesgo. Sin embargo, ha habido escasa confirmación de estos hallazgos a nivel internacional. El estudio actual evalúa el impacto de la mentoría por parte de estudiantes universitarios capacitados en autoestima y conexión escolar de los niños en comparación con los compañeros de escuela que no participaron en el programa. Mentor-UP es un programa de mentoría semanal llevado a cabo en la escuela y la comunidad que se implementó en el norte de Italia durante un período de siete meses. Los participantes (209 estudiantes, 34 en el grupo experimental y 175 en el grupo de comparación de edades comprendidas entre 11 y 13 años, 56% hombres, 27% inmigrantes) informaron de su nivel de autoestima y conexión escolar al principio y al final del programa. Los resultados mostraron un aumento significativo en la autoestima de los niños mentorizados en comparación con el grupo de control, mientras que la diferencia en la conexión escolar no fue significativa. Los hallazgos respaldan la efectividad de Mentor-UP para fomentar la autoestima de los jóvenes

Using Harter and Likert Response Formats in Middle Childhood: A Comparison of Attachment Measures.

Self-report questionnaires based on Harter's response format ("Some kids . . . but other kids . . . ") are commonly used in developmental and clinical research settings, but the reliability and validity of this format in middle childhood are still under debate. The current study aimed to test the psychometric proprieties of Harter versus Likert response formats as applied to two attachment questionnaires in a sample of 410 Italian children aged 8 to 10 years. Participants completed the Experiences in Close Relationships-Revised Child version (n = 102, 4-point Likert-type scale; n = 104, adapted Harter version) and the Security Scale (n = 95, Harter's format; n = 109 adapted 4-point Likert version). Results of multigroup confirmatory factor analyses indicated that the two response formats exhibited comparable reliability and factorial validity, although a slight superiority of Harter's format emerged for the Security Scale. External validity was supported for both answer formats. Implications for developmental theory and practice are discussed.

The interplay of reading-related emotions and updating in reading comprehension performance.

BACKGROUND: Reading comprehension can be considered the main learning activity. All learning experiences are infused with emotions; however, to date, few studies have focused on the role of emotional aspects in reading comprehension performance. The impact of emotions on academic achievement is thought to be mediated or moderated by cognitive aspects. Among them, working memory updating is an executive function that plays a crucial role in reading comprehension. AIMS: This study aimed to investigate the relationships between reading-related emotions and reading comprehension performance. We also consider the role that updating may play in these relationships. SAMPLE: Two hundred and eight 8th graders were involved in four sessions. METHOD: Students completed measures of achievement emotions specifically related to reading comprehension activity, updating, and reading comprehension performance. Gender and general cognitive ability were also considered as control variables. Mixed-effects models were used for statistical analyses. According to the Akaike information criterion (AIC; Akaike, 1974), we selected the most plausible model among a set of candidate models fitted to the same data. RESULTS: Results showed that activating-negative emotions (i.e., anxiety, anger, and shame), deactivating-negative emotions (i.e., boredom and hopelessness), and updating are related with reading comprehension performance. Moreover, the interaction between activating-negative emotions and updating also emerged. When activating-negative emotions interact with low and moderate updating, students' reading comprehension performance gets worse. CONCLUSIONS: The study indicates the moderating role of a main cognitive ability in the link between reading-related emotions and reading comprehension performance. Strategies can be taught to improve students' ability to self-regulate negative emotions and to update information in working memory.

Are children with developmental dyslexia all the same? A cluster analysis with more than 300 cases.

Reading is vital to every aspect of modern life, exacerbated by reliance of the internet, email, and social media on the written medium. Developmental dyslexia (DD) characterizes a disorder in which the core deficit involves reading. Traditionally, DD is thought to be associated with a phonological impairment. However, recent evidence has begun to suggest that the reading impairment in some individuals is provoked by a visual processing deficit. In this paper, we present WISC-IV data from more than 300 Italian children with a diagnosis of DD to investigate the manifestation of phonological and visual subtypes. Our results indicate the existence of two clusters of children with DD. In one cluster, the deficit was more pronounced in the phonological component, while both clusters were impaired in visual processing. These data indicate that DD may be an umbrella term that encompasses different profiles. From a theoretical perspective, our results demonstrate that dyslexia cannot be explained in terms of an isolated phonological deficit alone; visual impairment plays a crucial role. Moreover, general rather than specific accounts of DD are discussed.

Attachment and problematic Facebook use in adolescents: The mediating role of metacognitions.

BACKGROUND AND AIMS: Recent research used attachment theory and the metacognitive tenet as frameworks to explain problematic Facebook use (PFU). This study aims to test, in a single model, the role of different attachment styles and metacognitions in PFU among adolescents. METHODS: Two separate studies were conducted in order to establish the link between security (Study 1) and insecurity (Study 2), metacognitions, and PFU. A total of 369 and 442 Italian adolescents (age: 14-20 years old) participated in Study 1 and Study 2, respectively. RESULTS: Path analyses revealed the relative importance of different attachment dimensions with mother and father in predicting PFU and the mediating role of metacognitions between attachment styles and PFU. DISCUSSION AND CONCLUSION: In conclusion, since attachment styles and PFU may significantly affect adolescents' development and well-being, the results of this study may provide some practical indications for researchers and practitioners.

Self-Construals and Social Adjustment in Immigrant and Nonimmigrant Early Adolescents: The Moderating Role of Executive Functioning.

This study examined whether executive functions (EFs) moderate the association between independent and interdependent self-construals and social adjustment in 488 Moroccan, Romanian, and Italian preadolescents (ages 11-13) in Italy. Participants were assessed using self-report questionnaires and standardized EF tasks. Better working memory was related to increased social competence across all groups. High levels of inhibitory control were found to enhance the positive relation between interdependence and prosocial behavior for native Italian youth, and between interdependence and social competence for Moroccan preadolescents. High levels of cognitive flexibility boosted the interdependence-social competence link for the immigrant groups, whereas among native Italian preadolescents, the interdependence-social competence link was significant at low levels of flexibility. Implications for developmental theory and practice are discussed.

Inferential false memories for emotional events in older adults.

Emotional events have been shown to protect individuals against false memory when remembering scripted material. Whether the same is true also for older adults, however, is unclear, and it has been investigated in the present study. Seventy-six older adults (age range 65-89 years) were presented with a series of photographs depicting scripted events. Each event included the consequence of an action whose corresponding cause was not presented; the consequence was either neutral, negative, or positive. False recognitions of unseen causes of the consequences (i.e., 'causal errors') were calculated. A Bayesian inferential approach was adopted in order to include evidence from previous studies and to use it as a benchmark for newly collected information. Older adults showed enhanced false memories and lower accuracy than previously reported in studies on young adults. Compared to neutral ones, both negative and positive events were associated with fewer false memories in older adults. The emotional false memory 'profile' of older adults was very similar, in terms of the effect size, to the one previously found in young adults, although the overall chances of older adults incurring errors were higher. Results are discussed considering both cognitive and socioemotional hypotheses on emotional memory in late adulthood.

A Maximum Entropy Procedure to Solve Likelihood Equations.

In this article, we provide initial findings regarding the problem of solving likelihood equations by means of a maximum entropy (ME) approach. Unlike standard procedures that require equating the score function of the maximum likelihood problem at zero, we propose an alternative strategy where the score is instead used as an external informative constraint to the maximization of the convex Shannon's entropy function. The problem involves the reparameterization of the score parameters as expected values of discrete probability distributions where probabilities need to be estimated. This leads to a simpler situation where parameters are searched in smaller (hyper) simplex space. We assessed our proposal by means of empirical case studies and a simulation study, the latter involving the most critical case of logistic regression under data separation. The results suggested that the maximum entropy reformulation of the score problem solves the likelihood equation problem. Similarly, when maximum likelihood estimation is difficult, as is the case of logistic regression under separation, the maximum entropy proposal achieved results (numerically) comparable to those obtained by the Firth's bias-corrected approach. Overall, these first findings reveal that a maximum entropy solution can be considered as an alternative technique to solve the likelihood equation.